Interpreting Vague and Ambiguous Referring Expressions by Dynamically Binding to Properties of the Context Set

Referring expressions with vague and ambiguous modifiers, such as “a quick visit” and “the big meeting,” are difficult for computers to interpret because their words’ meanings are in part defined by context, which changes throughout the course of an interpretation. In this paper, we present an approach to interpreting context-dependent referring expressions that uses dynamic binding. During the incremental interpretation of a referring expression, a word’s meaning can be defined in part by properties from the current candidate referents—its denotation up to the previous word for the tentative interpretation. 1 Referring expressions in context For a hearer to understand the intended meaning of a speaker’s utterance, he must make inferences based not only on evidence in the utterance’s linguisticallyencoded surface meaning but also on outside information, collectively referred to as discourse context (for an introduction, read [1, Ch. 1]). Nowhere is this more evident than in linguistic reference—when a speaker attempts to use her utterance to convey the identity of some entities (or set of entities) to her audience. The speaker does so by producing a referring expression, namely: “a description of an entity [or entities] that enables the hearer to identify that entity in a given context” [2]. Consider the referring expression “it”: it seems evident that for the hearer to resolve what meaning the speaker intended by using the pronoun, he must draw from information outside of the pronoun itself. We have found that by focusing on the ubiquitous task of reference, large portions of context can be constrained so that others can be investigated. In general, it is the hearer’s job to use information from the context to determine which of a presumed set of meanings the speaker intended when she chose to use a particular lexical item (e.g., a morpheme, word, or idiom). And although the lexical items in an utterance arrive in a linear order, the hearer may need to backtrack and revise his decisions based on subsequent information. If the hearer does not revise an incorrect decision, he will likely fail to arrive at the speaker’s intended meaning. The linguistic phenomena of ambiguity and vagueness are ? We are grateful for the support from the sponsors of the MIT Media Lab. 2 Dustin A. Smith and Henry Lieberman two root causes of interpretive decision points. In certain contexts, these phenomena make linguistic communication efficient for humans [3, 4] and extremely challenging for computational models. Our goal is to build a computational model of reference that is able to represent only the relevant linguistic choices, and then make the correct decisions. We approach the problem in two stages: (1) making the system expressive enough to capture desired linguistic phenomena by ensuring the system is capable of representing all choice points that lead to the desired output and (2) finding a control algorithm that minimizes the number of choice points considered to produce the desired output. Unfortunately, theories of discourse context are rarely defined precisely; and so it is difficult to separate the components of context that influence lexical items from those that do not. We attempt to rectify this: in section 1.1, we summarize several ways context can influence the interpretation of referring expressions, and in section 1.3, describe a constrained communication task in which context’s influence on lexical items can be modeled directly. Afterward, we present an incremental model of reference interpretation that defines the meanings of vague (gradable) and lexically ambiguous adjectives using information from the on-line denotation. 1.1 What components of discourse context influence reference interpretation? In the study of language, the term “context” has been used to connote a wide range of information that is available to the speaker or hearer. A skeptic might take it to mean any information that outside the scope of the theory at hand. Tomasello described discourse context as “information that is available to [both speaker and hearer] in the environment, along with what is ‘relevant’ to the social interaction, that is, what each participant sees as relevant and knows that the other sees as relevant as well—and knows that the other knows this as well, and so on, potentially ad infinitum. This kind of shared intersubjective context is what we may call following [5] common ground...it takes [hearer and speaker] beyond their own egocentric perspective on things” [6, pp. 76]. Of course, common ground is a fiction: in addition to being paradoxically recursive, neither speaker nor hearer are omniscient so neither could ever know the true common ground. However, as a theoretical concept it may still be useful to envisage such an idealized state both the speaker and hearer’s inferential processes work toward in order to make the reference task succeed. As such, the speaker and hearer each have their own notions of success. The speaker wants to convey the referents to the hearer, so she must take into account what he knows or is capable of inferring. Similarly for the hearer, the speaker’s act of reference contains “an implicit assurance that he has enough information to uniquely identify the referent, taking into account the semantic content of the 1 Because our system is incremental, the “desired output” of a referring expression can be evaluated at intermediate stages. Interpreting Vague and Ambiguous Referring Expressions 3 referring expression and information from the context, whether situational (i.e. currently perceivable), linguistic, or mental (i.e. memory and knowledge)” [7]. For reference tasks, the knowledge speaker and hearer can be expected to have minimally includes: Task. The speaker and hearer’s shared tasks determine what is relevant and important to them, and thus their communication goals as well. Using the pragmatic theory of [8], the information needs of the task constitute its questions under discussion, which are a central impetus of communication. From our computational perspective, we take a referential question under discussion to be an unbounded typed variable in a plan. The question is answered when the variable is bound to a knowledge representation that meets certain type restrictions. Questions under discussion give rise to communication goals, which are fulfilled by communication acts toward these goals (e.g., speaking, gesturing). For reference tasks, the communication goal is at least in part referential: to make the intended referent(s) mutually known to hearer and speaker (i.e., in the common ground). Referential domain. Entities in the environment, which are mutually perceived, along with concepts from background knowledge constitute potential targets of referring expressions. Dialogue history. The speaker and hearer can be expected to remember the previous dialogue acts. For reference, this is especially important because after a speaker introduces a referent to discourse, she typically mentions it again—often using abbreviated referring expressions [9–11]. From a computational perspective, the referents in the dialogue history could be thought of as symbol table used by compilers and interpreters to map each symbol to its type, scope and value—namely, its location in memory. Instead of describing these contextual constraints individually, we will introduce an abstraction called the context set. It is a construct from theoretical linguistics that represents the “live options”—viable candidates for an interpretation process, which evolves over the course of dialogue [12]. It is the hypothesis space of interpretations. For utterances outside of reference, the concept of “what constitutes an interpretation” is difficult to pin down; however, for reference tasks, the context set can be seen as the referential domain plus all of its combinatoric possibilities. 1.2 Characterizing the two reference tasks The referential domain and its valid means of combination are both constrained by information from the task, dialogue history, and lexical-semantic knowledge. By constraining each of these elements, then, at least for reference tasks, we can replace the illusive concept of context with a single construct, the context set, which expresses the sum of all contextual constraints on the targets for interpretation. We will now attempt to formalize the broader communication tasks of the speaker and hearer. When referring, the speaker and hearer complete 4 Dustin A. Smith and Henry Lieberman two structurally similar tasks. The speaker completes a referring expression generation (REG) task: given an initial context set (defined in Section 1.4) and a designated member of it called the target set, she produces a referring expression which she expects will enable the hearer to infer her intended target set from the rest of the elements in the context set, called distractors [13]: REG(context set, target set)→ referring expression (1) A hearer completes a referring expression interpretation (REI) task: given a referring expression, his goal is to jointly infer the context set and the targets that the speaker intended: REI(referring expression)→ 〈context set, target set〉1 . . . 〈context set, target set〉n (2) Reference tasks do not always succeed. We define a reference failure as a mismatch between the speaker’s intended target set and the one (or ones) yielded by the hearer’s interpretation. If the referring expression leads the hearer to generate multiple plausible interpretations (e.g., n > 1), we call such referring expressions uncertain. In the next section, we describe some of the issues that lead to these uncertain referring expressions, which in turn commonly lead to reference failures. 1.3 Restricting the discourse context Although all of the aforementioned components of context can potentially impact reference interpretation [14, 11, 15], to avoid their influence we can restrict the task setting so that many aspects of context prior to an utterance are constrained. This allows us to investigate and model how words interact with context. We do this by: – using a referential domain that is co-present [16], which we achieved by using visual scenes. Most of the referring expressions described in this paper will be interpreted with respect to one of two referential domains, Circles and Kindles, which are expressed as co-present visual scenes and therefore assumed to be in the speaker and hearer’s common ground: Interpreting Vague and Ambiguous Referring Expressions 5 Fig. 1. The Amazon Kindle referential domain containing 5 referents: k1, k2, k3, k4 and k5.